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Fig. 1 




Fig. 2 



300 



Server identifies a document (or documents) accessed or 
viewed by a user using the user system 


302 








Server determines which sections of the document 
identified in step 302 are to be analyzed 


304 








Server extracts (or generates) UDIOs from the sections of 
the document determined in step 304 


306 








UDIOs extracted in step 306 are analyzed 


308 








Server selects CPIOs to be provided to the user 


310 








CPIOs selected in step 310 are communicated to the user 
system 


312 








User system outputs information contained in the selected 
CPIOs 


314 



Fig. 3 



308 



From step 306 in Fig. 3 




For each extracted UDIO, based upon the class of the 
UDIO {as determined in step 402), determine one or more 
content recognition techniques to be applied to the UDIO 



404 



For each extracted UDIO, determine the contents of the 
UDIO by applying the content recognition techniques for 
the UDIO determined in step 404 



406 



Store the UDIOs, their associated class information, and 
their content information 



408 



To step 310 in Fig. 3 



Fig. 4 



From step 308 in Fig. 3 



310 



Identify one or more selection techniques to be used for 
selecting CPIOs to be provided to the user 



502 



Apply each selection technique identified in step 502, to 
determine relevancy scores between the CPIOs and the 
UDIOs for each selection technique 



T 







Combine the results of the various selection techniques 

applied in step 504 to generate a single "final" list of 
selected CPIOs to be communicated to the user system 







To step 312 in Fig. 3 



Fig. 5 



From step 308 in Fig. 3 



602 



604 



Determine concepts of interest to 
a user 






For each UDIO, determine a 
degree of relevancy of the UDIO 
to each concept of interest to the 

user determined in step 602 








606 



Based upon the relevancy of the 
UDIOs and CPIOs to the user 
concepts of interest, for each 

CP 10, determine the degree of 
relevancy of the CPIO to each 

UDIO as it relates to a particular 
user concept of interest 



Determine one or more 
comparison techniques to be 
applied to each UDIO based upon 
the "class" of the UDfO 



For each UDIO, each comparison 
technique associated with UDIO is 

applied to determine relevancy 
between the UDIO and the CPIOs 



612 



Generate a single "final" list of 
CPIOs using a combination 
technique 



Select one or more CPIOs based 

upon the relevancy scores 
determined for the CPIOs in step 
606 



T 



616 



- 614 



Select one or more CPIOs based 

upon the relevancy scores 
associated with the CPIOs in step 
612 



To step 312 in Fig. 3 

Fig. 6 



^-706 



Parameters: 
Weighting Factors: (W1, W2, . . .)-^ 
^-Thresholds (T1.T2, . . .) 706-a 

706-b 



704- 



Source#1: ListofCPIOs _ 
and relevancy info 

Source #1: ListofCPIOs _ 
and relevancy info 

Source #1: ListofCPIOs — 
and relevancy info 



Combination 
Technique 



A single "final" 
listofCPIOs ^ 
708 
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Fig. 7 



Initialize variables: 
Array NumVotes{1..N] = 0 
Array WeightVotes[1..N] = 0 
Array of Lists Sources Voting[1 ..N] = Null 
#S = Total number of sources 
d = 1 




804 



(d>#S)? 



802 



Select source S H 



#L = Total number of ListElements in the list for source S d 
e = 1 




CPIO de = CPIO object in List£lement e of source S d 
R de = Relevance value in ListElement e of source S d for CPlO de 




"81 2 
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NumVotes[CPIOJ = NumVotes[CPIOJ + 1 
WeightVotes[CPIO de ] = WeightVotes[CPIOJ + W Sd 
Append S d to list SourcesVoting[CPIO de ] 



-818 



d = d + 1 
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Generate a "final" list of one or CPIOs based on specified 
critieria 



822 



To step 312 in Fig. 3 

Fig. 8 



